Hypothesis Testing Part 2
POLS 3316: Statistics for Political Scientists

##Today

• What is hypothesis testing?

    - Principles of hypothesis testing
    - The Standard Error
    - The first test: Z-scores

Part 1: Hypothesis Testing

• Last week:

• The *68-95-99.7 Rule: normal distribution probability shorthand

• Tying samples and populations

• The Central Limit Theorem: get to normal distribution

• The Law of Large Numbers: further tie sample to population

• Two new, related statistics: standard error and Z-score

• The goal was Hypothesis Testing

So what is hypothesis testing?

• We have a theory and we want to see if it’s valid
• We formulate hypotheses (pl.) that would be true if the theory was valid
• We try to disprove or falsify them
• How? By statistically testing data

Review: The Null and Alternative Hypotheses

• The Alternative Hypothesis: This is actually the hypothesis that agrees with our theory. Why?
• The default assumption is that our theory is that there is no effect: The Null Hypothesis
• We can’t prove our hypothesis, but we can get sufficient evidence to reject the null hypothesis
• How? Data and Statistics

How do we reject of confirm the null hypothesis?

How do we reject of confirm the null hypothesis?

• Conduct experiments or make observations to gather data

You know nothing

How do we reject of confirm the null hypothesis?

• Conduct experiments or make observations to gather data

You know nothing

John Snow and the Ghost Map

How do we reject of confirm the null hypothesis?

• Conduct experiments or make observations to gather data
• Compare the results to expectations of random chance

How do we reject of confirm the null hypothesis?

• Conduct experiments or make observations to gather data
• Compare the results to expectations of random chance
• Match random chance: Retain the null hypothesis, reject the alternative hypothesis

How do we reject of confirm the null hypothesis?

• Conduct experiments or make observations to gather data
• Compare the results to expectations of random chance
• Match random chance: Retain the null hypothesis, reject the alternative hypothesis
• Do not match random chance: Reject the null hypothesis - evidence in favor alternative hypothesis

How do we reject of confirm the null hypothesis?

• Conduct experiments or make observations to gather data
• Compare the results to expectations of random chance
• Match random chance: Retain the null hypothesis, reject the alternative hypothesis
• Do not match random chance: Reject the null hypothesis - evidence in favor alternative hypothesis
• How do we do compare to random chance? Tests based on probability and statistics

Test statistics

• Z-score
• t-test
• Chi-square
• ANOVA
• Many others

The Z-score

• Based on the standard normal distribution
• Application of the 69-95-99.7 rule

68-95-99.7 rule

The Z-score

• Based on the standard normal distribution
• Based on the 69-95-99.7 rule
• Measures how many standard errors a value is from the mean

The Z-score

• Based on the standard normal distribution
• Based on the 69-95-99.7 rule
• Measures how many standard errors a value is from the mean

Fast forward preview: The standard error is a special standard deviation the standard deviation of the sampling distribution of the mean.

The Z-score

• Based on the standard normal distribution

• Based on the 69-95-99.7 rule

• Measures how many standard errors a value is from the mean

• Can be used to test:

    - hypotheses about the mean of a population
    - hypotheses about the difference between two means (two groups with identical distributions)
    - hypotheses about the difference between a mean and a value
    - hypotheses about the difference between two proportions

Standard error: Tying samples to populations

• Terms

  • statistics - sample
  • parameter - population

• We want to use the sample mean/median/etc to estimate the population version of the same measurement

Standard error: Tying samples to populations

• Quantifies the range around population value (parameter) for the sample value (the statistic)

Standard error: Tying samples to populations

• Quantifies the range around population value (parameter) for the sample value (the statistic)
• Usually the mean but you can figure a standard error for other statistics like the median

Standard error: Tying samples to populations

• Quantifies the range around population value (parameter) for the sample value (the statistic)
• Usually the mean but you can figure a standard error for other statistics like the median
• Distance from the mean is a measure of…

Standard error: Tying samples to populations

• Quantifies the range around population value (parameter) for the sample value (the statistic)
• Usually the mean but you can figure a standard error for other statistics like the median
• Distance from the mean is a measure of…

Dispersion

Standard error: Tying samples to populations

• Quantifies the range around population value (parameter) for the sample value (the statistic)
• Usually the mean but you can figure a standard error for other statistics like the median
• Distance from the mean is a measure of…Dispersion
• If we want to measure disperstion in units equal to the mean, we want to measure dispersion with?

Measure dispersion

• If we want to measure disperstion in units equal to the mean, we want to measure dispersion with?

Standard deviation

The Standard Error of the Mean

• Standard deviation measures dispersion relative to the mean

The Standard Error of the Mean

• Standard deviation measures dispersion relative to the mean
• Standard error measures dispersion between the sample mean and the population mean

The Standard Error of the Mean

• Standard deviation measures dispersion relative to the mean
• Standard error measures dispersion between the sample mean and the population mean
• Standard error of the mean is the sample standard deviation divided by the square root of the sample size:

\(\frac{s}{\sqrt{n}}\)

Stanard Error

• Standard Error is the standard deviation of the sample means.

    - If we do 1000 trials of random, indendepent, identically distributed variables (random IID variables) from any distribution
    - The means of each trial are the sample means
    - Central Limit Theorem tells us that the distribution of the sample means will converge to a normal distribution 
    - If we made a vector of the means of the 10000 trials flipping a coin 20 times and plotted a histogram, it would look like this:

Stanard Error

Standard Error

• Theoretically, if we do an experiment with 500 subjects, it’s one trial with one sample mean. When we start doing hypothesis tests,
• Practically, we can’t do 10000 trials of 500 subjects each. We can only do one trial with 500 subjects, or…
• We use observational data with 500 observations
• We don’t need 500 data points, next week I’ll show you why 30 is sufficient in many cases

Z-Score: Concept

• Number of standard errors from the mean

Z-Score: Concept

• Number of standard errors from the mean
• Probability that actual population parameter is approximately equal to sample statistic

Z-Score: Concept

• Number of standard errors from the mean

• Probability that actual population parameter is approximately equal to sample statistic

• If we know the sample mean, \(\bar{x}\), is 50

• standard error, \(\sigma\), is 1

• We want to locate the population mean, \(\mu\)

Z-Score: Concept

68-95-99.7 Rule

• 99.7% probability that the true population mean is between \(\bar{x} \pm 3 * SE\) or 50 \(\pm\) 3 * SE
• If SE is 1..

Z-Score: Concept

68-95-99.7 Rule

• 99.7% probability that the true population mean is between \(\bar{x} \pm 3 * SE\) or 50 \(\pm\) 3 * SE
• If SE is 1..
• 99.7% probability that the true population mean is between 47 and 53.

Confidence Interval: Concept

68-95-99.7 Rule

• 99.7% probability that the true population mean is between \(\bar{x} \pm 3 * SE\) or 50 \(\pm\) 3 * SE

Confidence Interval

• The 99.7% Confidence Interval of the sample mean with a sample mean of 50 and standard error of 1 is 47 to 53.

Z-Score: Formula

• The Z-score gives us a formula which we can compare to standard tables of probabilities

Z-Score: Formula

• The Z-score gives us a formula which we can compare to standard tables of probabilities

• \(z = \frac{x - \mu}{\sigma}\)

Z-Score: Confidence Interval

• The Confidence Interval with the Z-Score is sample mean \(\pm\) the Margin of Error which we get from (just for illustration at this point):

Why Z-score instead of 68-95-99 Rule?

• 68-95-99.7 approximates 2 standard deviations for 95%
• Actual value of 2 sd is 95.45%
• Precise Z-value for 95% is 1.96
• Z-score of 2 is still a good mental shortcut for 95% (better than 95%)
• Journal articles publish outcomes with standard errors underneath

Z-scores journal articles

===============================================
                        Dependent variable:    
                    ---------------------------
                               dist            
-----------------------------------------------
speed                        3.932***          
                              (0.416)          
                                               
Constant                     -17.579**         
                              (6.758)          
                                               
-----------------------------------------------
Observations                    50             
R2                             0.651           
Adjusted R2                    0.644           
Residual Std. Error      15.380 (df = 48)      
F Statistic           89.567*** (df = 1; 48)   
===============================================
Note:               *p<0.1; **p<0.05; ***p<0.01

More on Z-Scores and Standard Errors

More on Z-scores and Standard Errors

https://www.statisticshowto.com/probability-and-statistics/z-score/

https://www.statisticshowto.com/probability-and-statistics/statistics-definitions/what-is-the-standard-error-of-a-sample/

https://www.investopedia.com/ask/answers/042415/what-difference-between-standard-error-means-and-standard-deviation.asp

https://www.investopedia.com/ask/answers/021115/what-difference-between-standard-deviation-and-z-score.asp

Authorship, License, Credits

• Author: Tom Hanna

• Website: tomhanna.me

• License: This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.</>